Skip to content

Enhances MSSQL connection with pyodbc support #4335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

mattiasthalen
Copy link

Adds support for pyodbc as an alternative driver for MSSQL connections, providing users with more flexibility and authentication options.

  • Introduces a driver option to choose between pymssql and pyodbc.
  • Implements connection string generation for pyodbc based on connection parameters.
  • Includes validation for different authentication methods when using pyodbc.
  • Updates dependencies to include pyodbc.

@mattiasthalen
Copy link
Author

@erindru hey, I just got around to do this after some onboarding at my new employer.

I also had some time to play around with the SQL Database object in fabric, and... I think just adding pyodbc allows us to connect to it like any other mssql. Dare I say it? It feels like it's really close to have some initial Fabric support! 👯

I know, I know, it's not the lakehouse, nor the warehouse objects. But it's something!

@mattiasthalen mattiasthalen force-pushed the add-pyodbc-option-to-mssql branch 3 times, most recently from 05f43a0 to 249f1f7 Compare May 8, 2025 20:47
@erindru
Copy link
Collaborator

erindru commented May 8, 2025

Thanks @mattiasthalen !

Yes we were waiting for Fabric Lakehouse to become a sane product before trying to support Fabric. Interesting to see if this actually works enough to hit the SQL Warehouse endpoint or if there are some Fabric-specific nuances.

Can you please add a section to the docs on how to set up pyodbc to work with Microsoft's ODBC driver?

Also, can you please rebase onto main and run make style? A bunch of test failures were related to code linting

(if this is still a WIP I can hold off reviewing until it leaves draft status)

@erindru erindru linked an issue May 8, 2025 that may be closed by this pull request
@mattiasthalen
Copy link
Author

@erindru I tried hitting the Warehouse endpoint, but there are several differences compared to MSSQL that doesn't work the same. E.g., information_schema is case sensitive.

Shouldn't be that hard to support it though. I'll probably have a look on that next.

Still refreshing to see that the SQL Database object was supported by just being able to login.

Ok, I was scratching my head on how to solve those failing tests 😅

Will fix asap and send for review 🎉

@mattiasthalen mattiasthalen force-pushed the add-pyodbc-option-to-mssql branch 2 times, most recently from 75522a9 to 58e5f22 Compare May 9, 2025 08:06
@mattiasthalen mattiasthalen marked this pull request as ready for review May 9, 2025 08:27
@mattiasthalen mattiasthalen force-pushed the add-pyodbc-option-to-mssql branch from dc299ef to c31c2f4 Compare May 9, 2025 19:06
@mattiasthalen mattiasthalen force-pushed the add-pyodbc-option-to-mssql branch from c31c2f4 to 064ace9 Compare May 10, 2025 10:20
@erindru
Copy link
Collaborator

erindru commented May 11, 2025

Thanks, this is looking pretty good!

Just one more thing - there are no tests. Can you please add some tests to test_connection_config that show:

  • The parameter validations you added are working
  • The _connection_kwargs_keys are as expected for each variant (pymssql and pyodbc)
  • pyodbc.connect gets invoked with the correct ODBC string

@mattiasthalen
Copy link
Author

Hey sorry for being slow to progress. It's been hectic at work, but I hopens have time this weekend and write the tests

@erindru
Copy link
Collaborator

erindru commented May 20, 2025

No problem, there is no expectation / time pressure on community contributions! Thanks for the update

@crazy-treyn
Copy link

crazy-treyn commented May 30, 2025

@erindru I tried hitting the Warehouse endpoint, but there are several differences compared to MSSQL that doesn't work the same. E.g., information_schema is case sensitive.

FYI, Fabric Warehouses can now be created with a case insensitive collation: Create Warehouse with Case Insensitive Collation

The new Fabric CLI makes it really easy to create a new Warehouse with this case insensitive collation without messing with the REST APIs.

It is possible that your implementation would work with a Warehouse created with a case insensitive collation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MS SQL authentification with the managed identity
4 participants